Ranging device

ABSTRACT

A ranging device includes: a binning processing unit configured to perform binning processing for one or more frequency distributions held in one or more frequency distribution generation units of frequency distribution generation units; a determination unit configured to determine whether a component of reflected light from an object is included in the frequency distribution subjected to the binning processing; a binning setting unit configured to change the number of the frequency distribution generation units to be subjected to the binning processing performed by the binning processing unit in accordance with a result of the determination by the determination unit; and a distance calculation unit configured to calculate a distance to the object based on a time interval corresponding to a reception time of the reflected light when the determination unit determines that a component of the reflected light is included in the frequency distribution subjected to the binning processing.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to a ranging device.

Description of the Related Art

Japanese Patent Application Laid-Open No. 2020-112443, Japanese PatentApplication Laid-Open No. 2021-103101, and Japanese Patent ApplicationLaid-Open No. 2021-050950 disclose a ranging device that measures adistance to an object by emitting light from a light emitting unit andreceiving light including reflected light from the object by a lightreceiving element. The ranging device disclosed in Japanese PatentApplication Laid-Open No. 2020-112443 may perform ranging such that theranging condition is changed while an imaging frame is formed. Examplesof the ranging conditions include the number of avalanche photodiodesand sampling frequency.

However, in the technique disclosed in Japanese Patent ApplicationLaid-Open No. 2020-112443, it is necessary to take a time close to oneframe period from the change of the ranging condition to the detectionof reflected light by receiving light again. Therefore, it may be aproblem that the delay time from the change of the ranging conditionuntil the ranging is performed according to the changed rangingcondition is large.

SUMMARY OF THE INVENTION

It is an object of the present disclosure to provide a ranging devicecapable of improving ranging accuracy while suppressing an increase indelay time.

According to a disclosure of the present specification, there isprovided a ranging device including: a time counting unit configured toperform time counting; a plurality of light receiving units eachconfigured to generate a signal including a pulse based on incidentlight and perform an operation of counting the pulse; a plurality offrequency distribution generation units arranged corresponding to theplurality of light receiving units and each configured to hold afrequency distribution including a light reception count value of thepulse acquired at predetermined time intervals in the time counting; abinning processing unit configured to perform binning processing for oneor more frequency distributions held in one or more frequencydistribution generation units of the plurality of frequency distributiongeneration units; a determination unit configured to determine whether acomponent of reflected light from an object is included in the frequencydistribution subjected to the binning processing; a binning setting unitconfigured to change the number of the frequency distribution generationunits to be subjected to the binning processing performed by the binningprocessing unit in accordance with a result of the determination by thedetermination unit; and a distance calculation unit configured tocalculate a distance to the object based on a time intervalcorresponding to a reception time of the reflected light when thedetermination unit determines that a component of the reflected light isincluded in the frequency distribution subjected to the binningprocessing.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a schematic configuration exampleof a ranging device according to a first embodiment.

FIG. 2 is a diagram schematically illustrating an operation of theranging device according to the first embodiment in one ranging period.

FIGS. 3A, 3B, 3C, and 3D are histograms schematically illustrating afrequency distribution of the light reception count values according tothe first embodiment.

FIG. 4 is a flowchart illustrating an operation of the ranging deviceaccording to the first embodiment in one frame period.

FIG. 5 is a flowchart illustrating an operation of peak determinationprocessing according to the first embodiment.

FIG. 6 is a flowchart illustrating peak determination processing for onepixel according to the first embodiment.

FIG. 7 is a diagram illustrating coordinates of each pixel according tothe first embodiment.

FIG. 8 is a diagram illustrating an example of an output of adetermination unit where the number of binning pixels is 1×1 accordingto the first embodiment.

FIG. 9 is a diagram illustrating an example of an output of a distancecalculation unit where the number of binning pixels is 1×1 according tothe first embodiment.

FIG. 10 is a diagram illustrating an example of an output of thedetermination unit where the number of binning pixels is 2×1 accordingto the first embodiment.

FIG. 11 is a diagram illustrating an example of an output of thedistance calculation unit where the number of binning pixels is 2×1according to the first embodiment.

FIG. 12 is a diagram illustrating an example of an output of thedistance calculation unit at the end of processing according to thefirst embodiment.

FIG. 13 is an example of a histogram of a pixel at coordinates (0, 0)according to the first embodiment.

FIG. 14 is an example of a histogram of a pixel at coordinates (0, 2)according to the first embodiment.

FIG. 15 is an example of a histogram of a pixel at coordinates (1, 2)according to the first embodiment.

FIG. 16 is an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (0, 2) and the pixel ofcoordinates (1, 2) according to the first embodiment.

FIG. 17 is an example of a histogram of a pixel at coordinates (2, 0)according to the first embodiment.

FIG. 18 is an example of a histogram of a pixel at coordinates (3, 0)according to the first embodiment.

FIG. 19 is an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (2, 0) and the pixel ofcoordinates (3, 0) according to the first embodiment.

FIG. 20 is a flowchart illustrating peak determination processing forone pixel according to a second embodiment.

FIG. 21 illustrates an example of a histogram of a pixel at coordinates(0, 2) according to the second embodiment.

FIG. 22 is an example of a histogram of a pixel at coordinates (1, 2)according to the second embodiment.

FIG. 23 is an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (0, 2) and the pixel ofcoordinates (1, 2) according to the second embodiment.

FIG. 24 is an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (0, 2) and the pixel ofcoordinates (1, 2) according to a comparative example of the secondembodiment.

FIG. 25 is a schematic view illustrating the overall configuration of aphotoelectric conversion device according to a third embodiment.

FIG. 26 is a schematic block diagram illustrating a configurationexample of a sensor substrate according to the third embodiment.

FIG. 27 is a schematic block diagram illustrating a configurationexample of a circuit substrate according to the third embodiment.

FIG. 28 is a schematic block diagram illustrating a configurationexample of one pixel of a photoelectric conversion unit and a pixelsignal processing unit according to the third embodiment.

FIGS. 29A, 29B, and 29C are diagrams illustrating an operation of anavalanche photodiode according to the third embodiment.

FIG. 30 is a schematic diagram of a photodetection system according to afourth embodiment.

FIGS. 31A and 31B are schematic diagrams of equipment according to afifth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings. In the drawings,the same or corresponding elements are denoted by the same referencenumerals, and the description thereof may be omitted or simplified.

First Embodiment

FIG. 1 is a block diagram illustrating a schematic configuration exampleof a ranging device 30 according to the present embodiment. The rangingdevice 30 includes a control unit 31, a light emitting unit 32, a timecounting unit 33, ranging processing units 34 a and 34 b, a binningprocessing unit 35, a binning setting unit 36, and an output unit 37.The ranging processing unit 34 a includes a light receiving unit 341 a,a frequency distribution generation unit 342 a, a determination unit 343a, and a distance calculation unit 344 a. The ranging processing unit 34b includes a light receiving unit 341 b, a frequency distributiongeneration unit 342 b, a determination unit 343 b, and a distancecalculation unit 344 b. Note that the configuration of the rangingdevice 30 illustrated in the present embodiment is an example, and isnot limited to the illustrated configuration.

The ranging device 30 measures a distance to an object 40 by using atechnique such as light detection and ranging (LiDAR). The rangingdevice 30 measures the distance from the ranging device 30 to the object40 based on a time difference until the light emitted from the lightemitting unit 32 is reflected by the object 40 and received by the lightreceiving units 341 a and 341 b.

The light received by light receiving elements of the light receivingunits 341 a and 341 b includes ambient light such as sunlight inaddition to the reflected light from the object 40. For this reason, theranging device 30 measures incident light at each of a plurality of timeintervals, and performs ranging in which the influence of ambient lightis reduced by using a method of determining that reflected light isincident during a period in which the amount of light peaks. The rangingdevice 30 of the present embodiment may be a flash LiDAR that emitslaser light to a predetermined ranging area including the object 40, andreceives reflected light by a pixel array. The pixel array includes aplurality of light receiving units arranged two-dimensionally. Each ofthe plurality of light receiving units may be referred to as a pixel.

In the present embodiment, in order to simplify the description, it isassumed that the number of pixels of the pixel array is 16 pixels intotal, with 4 pixels in the horizontal direction (X direction) and 4pixels in the vertical direction (Y direction), but it is not limited tothis and may be changed accordingly. The X coordinates and the Ycoordinates of these pixels are from (0, 0) to (3, 3). The rangingprocessing unit 34 a is arranged corresponding to, among 16 pixels, apixel whose X coordinate and Y coordinate are (0, 0). The rangingprocessing unit 34 b is arranged corresponding to, among 16 pixels, apixel whose X coordinate and Y coordinate are (1, 0). Although notillustrated in FIG. 1 , ranging processing units corresponding to theother pixels are arranged in the same manner as the ranging processingunits 34 a and 34 b in the ranging device 30.

The ranging device 30 of the present embodiment can perform binningprocessing in which a plurality of pixels in the pixel array arecollectively handled as one pixel. Thereby, although the spatialresolution in the light-receiving surface of the pixel array decreases,the sensitivity can be improved because the light-receiving area can beenlarged in a pseudo-manner. In general, by performing binningprocessing for n pixels, a random component of noise is reduced to 1/√n.

The light emitting unit 32 is a light source that emits light such aslaser light to the outside of the ranging device 30. When the rangingdevice 30 is a flash LiDAR, the light emitting unit 32 may be a surfacelight source such as a surface emitting laser.

The time counting unit 33 performs time counting based on the control ofthe control unit 31, and acquires an elapsed time from a time at whichcounting is started as a digital signal. The control unit 31synchronously controls an emission timing of the light emitting unit 32and a timing at which the time counting unit 33 starts the timecounting. The time counting unit 33 counts the elapsed time from thelight emission by increasing the time count value by one each time acertain time elapses after the start of counting. The time counting unit33 includes, for example, a circuit such as a ring oscillator and acounter, and counts a clock pulse that vibrates at high speed and at aconstant period, thereby performing the time counting. For example, whenthe cycle of the clock pulse is one nanosecond and the time count valueis counted up from “0”, which is an initial value, to “10”, it isunderstood that the elapsed time from the start of the time counting is10 nanoseconds by referring to the time count value.

Each of the light receiving units 341 a and 341 b receives lightincluding reflected light emitted from the light emitting unit 32 andreflected by the object 40. Each of the light receiving units 341 a and341 b generates a pulse signal including a pulse based on the incidentlight. The light receiving units 341 a and 341 b are, for example, aphotoelectric conversion device including an avalanche photodiode as aphotoelectric conversion element. In this case, when one photon isincident on the avalanche photodiode and a charge is generated, onepulse is generated by avalanche multiplication. However, the lightreceiving units 341 a and 341 b may include, for example, photoelectricconversion elements using other photodiodes.

Note that a plurality of photoelectric conversion elements may bearranged in one light receiving unit, and pulses based on incident lighton the plurality of photoelectric conversion elements may be integratedand counted as a count value of the same pixel. This processing isreferred to as pixel binning processing. Although the spatial resolutionin the light receiving surface of the pixel array decreases byperforming the pixel binning processing, the probability that the pixelreceives the reflected light can be improved, and a distance in whichthe ranging can be performed is extended.

The frequency distribution generation units 342 a and 342 b, thedetermination units 343 a and 343 b, the distance calculation units 344a and 344 b, the binning processing unit 35, the binning setting unit36, and the output unit 37 are signal processing circuits that performsignal processing on the pulse signals output from the light receivingunits 341 a and 341 b. The signal processing circuits may include acounter for counting pulses, a processor for performing arithmeticprocessing of digital signals, a memory for storing digital signals, andthe like. The memory may be, for example, a semiconductor memory. Thecontrol of the operation timings and the like of each unit in the signalprocessing circuits may be performed by a control circuit (notillustrated), or may be performed by the control unit 31.

The frequency distribution generation units 342 a and 342 b are memoriesthat store the number of input pulses (light reception count value)divided for each time interval. Since each of the plurality of timeintervals corresponds to one interval of a histogram of the lightreception count values, the time intervals are sometimes referred to asbins. When receiving pulses from the light receiving units 341 a and 341b, the frequency distribution generation units 342 a and 342 b acquiretime count values output by the time counting unit 33. The time countvalues correspond to times at which a photon enter the light receivingunits 341 a and 341 b. The frequency distribution generation units 342 aand 342 b identify a bin corresponding to the acquired time count value,that is, the reception time. For example, when the time count value iscounted up in units of 1 nanosecond and the bins are divided in units of10 nanoseconds, the number of the bin at the light reception time is1/10 of the time count value. In this way, the frequency distributiongeneration units 342 a and 342 b can identify a bin corresponding to thetime count value. Next, the frequency distribution generation units 342a and 342 b increase the light reception count value of the specifiedbin by one. Through such processing, the frequency distributiongeneration units 342 a and 342 b generate frequency distributionsincluding the light reception count values of the plurality of bins.

The binning processing unit 35 acquires a setting value of the number ofpixels to be subjected to the binning processing (the number of binningpixels) from the binning setting unit 36. The binning processing unit 35performs binning processing on the frequency distributions output fromthe frequency distribution generation units 342 a and 342 b based on thesetting value. In a specific example of the ranging described below, inorder to simplify the description, it is assumed that the number ofbinning pixels can be set to either 1×1 or 2×1. Here, the notation “m×n”of the number of binning pixels indicates that the range of pixels to besubjected to the binning processing is m pixels in the X direction and npixels in the Y direction. It should be noted that the 1×1 setting canbe referred to as a condition for outputting the frequency distributionas it is without performing the binning processing because the range ofthe binning processing is only one pixel.

When the number of binning pixels is 1×1, the binning processing unit 35outputs the frequency distributions output from the frequencydistribution generation units 342 a and 342 b to the determination units343 a and 343 b as they are. When the number of binning pixels is 2×1,the frequency distributions output from the frequency distributiongeneration units 342 a and 342 b are subjected to the binning processingand output to the determination units 343 a and 343 b. The binningprocessing is specifically a processing of summing the light receptioncount values for each bin with respect to a plurality of inputtedfrequency distributions.

The determination units 343 a and 343 b determine whether or not acomponent of reflected light is included in the frequency distributionoutput from the binning processing unit 35, and output determinationresults. The processing of determining the reflected light may be, forexample, determining that a component of the reflected light is includedin the frequency distribution when there is a bin whose light receptioncount value exceeds a predetermined threshold value, and determiningthat a component of the reflected light is not included in the frequencydistribution when there is no bin whose light reception count valueexceeds the predetermined threshold value. The threshold value used forthis determination may be a predetermined constant value, or may be avalue obtained by averaging the light reception count values of the binsand adding a constant value. Since the S/N ratio improves as the numberof binning pixels increases, appropriate determination can be made evenwhen the threshold value is set low. Therefore, the threshold value maybe changed based on the number of binning pixels.

When determining that the component of the reflected light is includedin the frequency distribution, the determination units 343 a and 343 boutput information indicating a bin corresponding to the reception timeof the reflected light to the distance calculation units 344 a and 344b, respectively. When there are a plurality of bins whose lightreception count value exceeds the threshold value, informationindicating a bin whose light reception time is the earliest may beoutput. In this case, bins with close distances are prioritized.Alternatively, when there are a plurality of bins whose light receptioncount value exceeds the threshold value, information indicating a binwhose light reception count value is the largest or a bin whosedifference between the light reception count value and the thresholdvalue is the largest may be output. In this case, a bin with strongreflected light is prioritized. The priority of these bins may beappropriately set depending on the application.

The distance calculation units 344 a and 344 b calculate and output adistance to the object 40 based on the number of binning pixels outputfrom the binning setting unit 36, the determination result of thereflected light output from the determination units 343 a and 343 b, andthe bin corresponding to the reception time of the reflected light. Forexample, when the component of the reflected light is included in thefrequency distribution, the distance is calculated from the bincorresponding to the reception time and the calculated distance isoutput. For example, the distance calculation units 344 a and 344 b maycalculate the distance from the ranging device 30 to the object 40 usingthe expression “(bin number of reception time)×(length of time intervalper bin)×(speed of light)/2”. When the number of binning pixels hasreached the upper limit and the reflected light cannot be detected, thedistance calculation units 344 a and 344 b output information indicatingthat the distance cannot be calculated. The output unit 37 outputs thedistance information output from the ranging processing units 34 a and34 b corresponding to each pixel to the outside of the ranging device30.

The binning setting unit 36 outputs the setting of the number of binningpixels to the binning processing unit 35. The binning setting unit 36calculates a minimum number of binning pixels determined to include acomponent of reflected light in the frequency distribution whilechanging the number of binning pixels from 1×1 to the upper limit valueof the number of binning pixels that can be subjected to the binningprocessing.

FIG. 2 is a diagram illustrating an outline of the operation of theranging device 30 according to the present embodiment in one rangingperiod. In the description of FIG. 2 , it is assumed that the rangingdevice 30 is a flash LiDAR. In the “ranging period” of FIG. 2 , aplurality of frame periods FL1, FL2, . . . , FL3 included in one rangingperiod are illustrated. The frame period FL1 indicates a first frameperiod in one ranging period, the frame period FL2 indicates a secondframe period in one ranging period, and the frame period FL3 indicates alast frame period in one ranging period. The frame period is a period inwhich the ranging device 30 performs one ranging and outputs a signalindicating a distance (ranging result) from the ranging device 30 to theobject 40 to the outside.

In the “frame period” of FIG. 2 , a plurality of shots SH1, SH2, . . . ,SH3 included in the frame period FL1 and a peak output OUT areillustrated. The shot is a period in which the light emitting unit 32emits light once and a frequency distribution is generated by a lightreception count value based on the light emission. The shot SH1indicates a first shot in the frame period FL1. The shot SH2 indicates asecond shot in the frame period FL1. The shot SH3 indicates a last shotin the frame period FL1. The peak output OUT indicates a period duringwhich a ranging result is output based on a peak acquired byaccumulating signals of a plurality of shots.

In the “shot” of FIG. 2 , a plurality of bins BN1, BN2, . . . , BN3included in the shot SH1 are illustrated. “bin” indicates one timeinterval during which a series of pulse counting is performed, and is aperiod during which a light reception count value is acquired bycounting pulses. The bin BN1 indicates the fastest bin in the shot SH1.The bin BN2 indicates a next bin of bin BN1 in the shot SH1. The bin BN3indicates a last bin in the shot SH1.

The “time counting” in FIG. 2 schematically illustrates a pulse PL1 usedfor time counting in the time counting unit 33 in the bin BN1. Asillustrated in FIG. 2 , the time counting unit 33 generates a time countvalue by counting the pulse PL1 that rises periodically. When the timecount value reaches a predetermined value, the bin BN1 ends, and theprocess transitions to the next bin BN2.

FIGS. 3A to 3D are histograms schematically illustrating frequencydistributions of light reception count values generated by the frequencydistribution generation units 342 a and 342 b. FIGS. 3A, 3B, and 3Cillustrate examples of histograms of light reception count values(corresponding to the number of photons) in the first shot, the secondshot, and the third shot, respectively. FIG. 3D illustrates an exampleof a histogram acquired by integrating the number of photons of allshots. The horizontal axis represents a bin number, which corresponds toan elapsed time from light emission. An interval of the histogramcorresponds to a period of one bin in which photon detection isperformed. The vertical axis represents the light reception count valuedetected in each bin period. In these histograms, the bin number of theearliest bin is “0”, and the earliest bin may be referred to as a zerothbin and the next bin may be referred to as a first bin.

As illustrated in FIG. 3A, in the first shot, the light reception countvalue of the fifth bin BN11 is a peak. As illustrated in FIG. 3B, in thesecond shot, the number of photons of the second bin BN12 is equal tothe number of photons of the fourth bin BN13, and these are peaks. Asillustrated in FIG. 3C, in the third shot, the number of photons of thefifth bin BN14 is a peak. In the second shot, different bins from theother shots are peaks. This is due to light reception count values dueto ambient light other than the reflected light from the object 40.

As illustrated in FIG. 3D, in the histogram obtained by integrating thenumber of photons of all shots, the fifth bin BN15 is a peak. In thepeak output OUT illustrated in FIG. 2 , the time information of the bincorresponding to the peak of the integrated frequency distribution isoutput. The distance between the ranging device 30 and the object 40 canbe calculated from the time information.

By integrating the pulse count values of a plurality of shots, it ispossible to detect a bin having a high possibility of reflected lightfrom the object 40 more accurately even when pulse count by ambientlight is included as in the second shot illustrated in FIG. 3B.Therefore, even when the light emitted from the light emitting unit 32is weak, the ranging can be performed with high accuracy by employing aprocess in which a plurality of shots are repeated. In the flash LiDAR,in some cases, it is required to lower the intensity of the emittedlight in order to ensure safety of the eyeball depending on the useenvironment. Thus, even when the intensity of the reflected light is lowand the influence of the ambient light is high, the ranging accuracy canbe ensured by using a method of integrating a plurality of shots asdescribed above. Further, when the number of pixels of the pixel arrayis large in order to improve the resolution, the area of thephotoelectric conversion element becomes small and the sensitivity maybe reduced, but the ranging accuracy can be ensured by using a method ofintegrating a plurality of shots as described above. Further, in thepresent embodiment, the ranging accuracy can also be ensured byperforming the binning processing if necessary.

FIG. 4 is a flowchart illustrating an operation of the ranging device 30according to the present embodiment in one frame period. FIG. 4illustrates the operation from the start to the end of one frame period.In FIG. 4 , a loop from step S12 to step S15 indicates a process inwhich signal acquisition of one shot is performed. A loop from step S11to step S16 indicates a process in which signal acquisition from thefirst shot to the last shot is performed.

In the step S11, the light emitting unit 32 emits light to the rangingarea. At the same time, the time counting unit 33 starts time counting.Thereby, the signal acquisition processing of one shot is started. Thecontrol unit 31 synchronously controls the light emission of the lightemitting unit 32 and the start of counting by the time counting unit 33.Thus, the elapsed time from the light emission can be counted.

In the step S12, when the light is detected by the light receiving units341 a and 341 b (YES in the step S12), the process proceeds to step S13.When no light is detected (NO in the step S12), the process proceeds tothe step S15.

In the step S13, the frequency distribution generation units 342 a and342 b identify a bin corresponding to the light reception time from thetime count value output from the time counting unit 33. Then, in stepS14, the frequency distribution generation units 342 a and 342 bincrease the light reception count value of the identified bin by one.In the step S15, the frequency distribution generation units 342 a and342 b determine whether or not the detection of light reception of thelast bin is completed based on the time count value. When the detectionof light reception of the last bin is completed (YES in the step S15),the process proceeds to the step S16 to complete the operation of theshot. When the detection of light reception of the last bin has not beencompleted (NO in the step S15), the process proceeds to the step S12 andthe detection of light in the shot is continued.

In the step S16, when the shot completed immediately before is the lastshot (YES in the step S16), the process proceeds to step S17. When theshot completed immediately before is not the last shot (NO in the stepS16), the process proceeds to the step S12 to start the operation of thenext shot.

In the step S17, the binning processing unit 35, the determination units343 a and 343 b, the binning setting unit 36, and the distancecalculation units 344 a and 344 b perform peak determination processing.The peak determination processing will be described with reference toFIGS. 5 and 6 .

FIG. 5 is a flowchart illustrating an operation of peak determinationprocessing according to the present embodiment. FIG. 5 illustrates thepeak determination processing in the step S17 of FIG. 4 in more detail.

The peak determination processing in step S17 includes a loop processfrom step S21 to step S23. In the step S21, one pixel to be subjected topeak determination is selected from the plurality of pixels. In stepS22, peak determination processing is performed on the selected pixel.When the processing of the selected pixel ends in the step S23, a pixelto be subjected to the processing is shifted to the next pixel. Thisloop process is continued until the processing of all the pixels iscompleted. In this way, the peak determination processing in the stepS17 is sequentially executed for each of the plurality of pixels. Theorder of the processing is not particularly limited, and for example,the processing may be executed sequentially from the upper left pixel tothe lower right pixel. Further, the determination processing of the stepS22 may be performed in parallel for a plurality of pixels.

FIG. 6 is a flowchart illustrating a peak determination processing forone pixel according to the present embodiment. FIG. 6 illustrates inmore detail the peak determination processing for one pixel in the stepS22 of FIG. 5 . In the example of FIG. 6 , the initial value of thenumber of binning pixels is set to a minimum 1×1 pixel (no binningprocessing), and the number of binning pixels is increased stepwiseaccording to the detection result of the reflected light to obtain theminimum number of binning pixels within a range in which the distancecan be calculated. However, the method of setting the number of binningpixels is not limited to this. For example, a method of starting theinitial value of the number of binning pixels from the maximum value andreducing the number of binning pixels according to the detection resultof the reflected light may be applied. Alternatively, a method ofobtaining an appropriate value of the number of binning pixels by abisection method may be applied.

In step S31, the binning setting unit 36 initializes the number ofbinning pixels to 1×1. The binning setting unit 36 outputs the settingof the number of binning pixels to the binning processing unit 35.

In step S32, the binning processing unit 35 performs binning processingon the frequency distribution according to the number of binning pixelsoutput from the binning setting unit 36. When the number of binningpixels is 1×1, the binning processing unit 35 outputs the frequencydistribution of the light reception count value of each bin of the pixelto be processed at coordinates (X, Y). That is, the binning processingunit 35 outputs the frequency distribution as it is without performingthe binning processing.

When the number of binning pixels is 2×1, the binning processing unit 35obtains a set of coordinates (Xi, Yi) that satisfy the conditions offloor(X/2)=floor(Xi/2) and floor(Y/1)=floor(Yi/1) with respect to thecoordinates (X, Y) of the pixels. Then, the binning processing unit 35adds the light reception count value of each bin for all the pixels ofthe coordinates included in the set of coordinates (Xi, Yi), and outputsit as a frequency distribution of the coordinates (X, Y). Note that,“floor(x)” is a function that returns a maximum integer equal to or lessthan x (a floor function). For example, in the case where the number ofbinning pixels is 2×1, when the above-described calculation is performedfor the coordinates (0, 0) and (1, 0), floor(0/2)=floor(1/2) andfloor(0/1)=floor(0/1). Therefore, the frequency distribution of thecoordinates (0, 0) and the frequency distribution of the coordinates (1,0) are added and output. Similarly, in the case where the number ofbinning pixels is 1×3, a set of coordinates (Xi, Yi) is obtained so thatfloor(X/1)=floor(Xi/1) and floor(Y/3)=floor(Yi/3) with respect to thecoordinates (X, Y) of the pixel. The binning processing unit 35 outputsa frequency distribution obtained by adding the light reception countvalue of each bin for three coordinates included in the set ofcoordinates (Xi, Yi). It should be noted that it may be possible to setfor each pixel whether or not to apply the binning processing accordingto the above rule. For example, some pixels may be set so that thebinning processing is not performed thereto, or all pixels may be set sothat the binning processing is performed thereto according to the aboverule.

In step S33, each of the determination units 343 a and 343 b determineswhether or not a component of reflected light is included in thefrequency distribution output from the binning processing unit 35, andoutputs a determination result. When it is determined that there isreflected light in the pixel being processed (YES in the step S33), theprocess proceeds to step S37. When it is determined that there is noreflected light in the pixel being processed (NO in the step S33), theprocessing proceeds to step S34.

In the step S34, the binning setting unit 36 determines whether or notthe current number of binning pixels is less than a maximum value. Whenthe number of binning pixels is less than the maximum value (YES in thestep S34), the process proceeds to step S35. When the number of binningpixels is the maximum value (NO in the step S34), the process proceedsto step S36. When the number of binning pixels is either 1×1 or 2×1, itis determined that it is less than the maximum value when the number ofbinning pixels is 1×1, and it is determined that it is not less than themaximum value when the number of binning pixels is 2×1.

In the step S35, the binning setting unit 36 increases the number ofbinning pixels. After that, the process returns to the step S32, thebinning processing is performed again by the increased number of binningpixels, and the reflected light is detected again. This processing maybe, for example, changing the setting of the number of binning pixelsfrom 1×1 to 2×1.

When the number of settings of the number of binning pixels is three ormore, the order in which the number of binning pixels is increased threeor more times by repeating the loop from the step S32 to the step S35 isnot particularly limited. For example, the number of pixels in the Xdirection and the number of pixels in the Y direction may be alternatelyincreased. In this method, the number of binning pixels is increased inthe order of 1×1, 2×1, 2×2, 3×2, . . . . Alternatively, the number ofpixels in the Y direction may be increased after the number of pixels inthe X direction is increased a plurality of times. In this method, thenumber of binning pixels is increased in the order of 1×1, 2×1, 3×1,3×2, . . . .

In the step S36, since the reflected light cannot be detected even whenthe number of binning pixels is set to the maximum value, the distancecalculation units 344 a and 344 b output information indicating that thedistance cannot be calculated. The processing of outputting thisinformation may be, for example, processing of outputting a flagindicating that the distance cannot be calculated, processing ofoutputting a maximum value of the distance, processing of outputting asignal indicating that the distance is infinity, or the like, but theprocessing is not limited thereto. The information indicating that thedistance cannot be calculated may be collectively output using thecoordinates after the binning processing, or may be output for eachcoordinate when the number of binning pixels is 1×1, as in the exampledescribed later.

In the step S37, since the reflected light can be detected, the distancecalculation units 344 a and 344 b calculate and output the distance fromthe bin corresponding to the reception time of the reflected light.

Next, an example of the determination result of the reflected lightgenerated by the peak determination processing of FIG. 6 and an outputsignal of the distance output unit will be described with reference toFIGS. 7 to 19 . FIG. 7 is a diagram illustrating coordinates of eachpixel according to the present embodiment. As illustrated in FIG. 7 ,the following description will be given assuming that 16 pixels arearranged in four rows and four columns from coordinates (0, 0) tocoordinates (3, 3). In this description, it is assumed that the numberof binning pixels can be set to either 1×1 or 2×1.

First, the peak determination process of FIG. 6 for the coordinates (0,0) will be described. In the step S31, the number of binning pixels isinitialized to 1×1. In the step S32 immediately after that, since thenumber of binning pixels is 1×1, the frequency distribution of the pixelat the coordinates (0, 0) is output as it is. FIG. 13 is an example of ahistogram of the pixel at coordinates (0, 0) according to the presentembodiment. In the example of FIG. 13 , since the light reception countvalue of the first bin BN21 exceeds a threshold value TH1, it isdetermined in the step S33 that reflected light is included. FIG. 8 is adiagram illustrating an example of outputs of the determination units343 a and 343 b where the number of binning pixels is 1×1 according tothe present embodiment. As illustrated in FIG. 8 , the determinationoutput of the coordinates (0, 0) is “1” indicating that reflected lightis included. Therefore, the process proceeds to the step S37. In theexample of FIG. 13 , assuming that the time intervals of the binscorrespond to a distance of 20 m, the distance corresponding to the binBN21 exceeding the threshold value TH1 is 20 m.

In the step S37, the distance calculation units 344 a and 344 bcalculate a distance corresponding to the bin BN21 that exceeds thethreshold value TH1, and output the calculated distance as a distance ofthe coordinates (0, 0). FIG. 9 is a diagram illustrating an example ofoutputs of the distance calculation units 344 a and 344 b where thenumber of binning pixels is 1×1 according to the present embodiment.FIG. 12 is a diagram illustrating an example of outputs of the distancecalculation units 344 a and 344 b at the time of end of processingaccording to the present embodiment. As illustrated in FIGS. 9 and 12 ,20 m is output as a distance at coordinates (0, 0).

Next, the peak determination process of FIG. 6 for the coordinates (0,2) will be described. In the step S31, the number of binning pixels isinitialized to 1×1. In the step S32 immediately after that, since thenumber of binning pixels is 1×1, the frequency distribution of the pixelat the coordinates (0, 2) is output as it is. FIG. 14 is an example of ahistogram of the pixel at coordinates (0, 2) according to the presentembodiment. In the example of FIG. 14 , since there is no bin exceedingthe threshold value TH1, it is determined in the step S33 that reflectedlight is not included. As illustrated in FIG. 8 , the determinationoutput of the coordinates (0, 2) is “0” indicating that the reflectedlight is not included. Therefore, the process proceeds to the step S34.

In the step S34, since the number of binning pixels is 1×1, it isdetermined that the number of binning pixels is less than the maximumvalue. Therefore, the process proceeds to the step S35. In the step S35,the number of binning pixels is changed to 2×1, and the process returnsto the step S32.

In the step S32, since the number of binning pixels is 2×1, thefrequency distributions are added for the pixels of the coordinates (0,2) and (1, 2) and output. FIG. 14 illustrates an example of a histogramof the pixel at coordinates (0, 2) according to the present embodiment,and FIG. 15 illustrates an example of a histogram of the pixel atcoordinates (1, 2) according to the present embodiment. FIG. 16illustrates an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (0, 2) and the pixel ofcoordinates (1, 2) according to the present embodiment. That is, FIG. 16is a sum of the histogram of FIG. 14 and the histogram of FIG. 15 . Inthe example of FIG. 16 , since the light reception count value of thefifth bin BN22 exceeds the threshold value TH1, it is determined in thestep S33 that reflected light is included. FIG. 10 is a diagramillustrating an example of outputs of the determination units 343 a and343 b where the number of binning pixels is 2×1 according to the presentembodiment. As illustrated in FIG. 10 , the determination output of thecoordinates (0, 2) is “1” indicating that reflected light is included.Therefore, the process proceeds to the step S37. In the example of FIG.16 , assuming that the time intervals of the bins correspond to adistance of 20 m, the distance corresponding to the bin BN22 exceedingthe threshold value TH1 is 100 m.

In the step S37, the distance calculation units 344 a and 344 bcalculate a distance corresponding to the bin BN22 that exceeds thethreshold value TH1, and output the calculated distance as a distance ofthe coordinates (0, 2). FIG. 11 is a diagram illustrating an example ofoutputs of the distance calculation units 344 a and 344 b where thenumber of binning pixels is 2×1 according to the present embodiment.FIG. 12 is a diagram illustrating an example of outputs of the distancecalculation units 344 a and 344 b at the time of end of processingaccording to the present embodiment. As illustrated in FIGS. 11 and 12 ,100 m is output as a distance at coordinates (0, 2).

Next, the peak determination process of FIG. 6 for the coordinates (2,0) will be described. In the step S31, the number of binning pixels isinitialized to 1×1. In the step S32 immediately after that, since thenumber of binning pixels is 1×1, the frequency distribution of thepixels at the coordinates (2, 0) is output as it is. FIG. 17 is anexample of a histogram of the pixel at coordinates (2, 0) according tothe present embodiment. In the example of FIG. 17 , since there is nobin exceeding the threshold value TH1, it is determined in the step S33that reflected light is not included. As illustrated in FIG. 8 , thedetermination output of the coordinates (2, 0) is “0” indicating thatthe reflected light is not included. Therefore, the process proceeds tothe step S34.

In the step S34, since the number of binning pixels is 1×1, it isdetermined that the number of binning pixels is less than the maximumvalue. Therefore, the process proceeds to the step S35. In the step S35,the number of binning pixels is changed to 2×1, and the process returnsto the step S32.

In the step S32, since the number of binning pixels is 2×1, thefrequency distributions are added for the pixels of the coordinates (2,0) and (3, 0) and output. FIG. 17 illustrates an example of a histogramof the pixel at coordinates (2, 0) according to the present embodiment,and FIG. 18 illustrates an example of a histogram of the pixel atcoordinates (3, 0) according to the present embodiment. FIG. 19illustrates an example of a histogram obtained by performing binningprocessing on the pixel of coordinates (2, 0) and the pixel ofcoordinates (3, 0) according to the present embodiment. That is, FIG. 19is a sum of the histogram of FIG. 17 and the histogram of FIG. 18 . Inthe example of FIG. 19 , since there is no bin exceeding the thresholdvalue TH1, it is determined in the step S33 that reflected light is notincluded. As illustrated in FIG. 10 , the determination output of thecoordinates (2, 0) is “0” indicating that the reflected light is notincluded. Therefore, the processing proceeds to the step S34.

In the step S34, since the number of binning pixels is 2×1, it isdetermined that the number of binning pixels is not less than themaximum value (the number of binning pixels is the maximum value).Therefore, the process proceeds to the step S36. In the step S36, sincethe reflected light cannot be detected even when the number of binningpixels is set to the maximum value, the distance calculation units 344 aand 344 b output information indicating that the distance cannot becalculated. The coordinates (2, 0) in FIGS. 11 and 12 indicate that thereflected light cannot be detected, and thus an infinity symbol (∞) isexpressed.

In the above example, the number of binning pixels can be set to either1×1 or 2×1. However, the number of binning pixels may be set to a valuelarger than 2×1 pixels. In this case, similar processing can be realizedby repeatedly executing the processing from the step S32 to the step S35to increase the number of binning pixels to a value larger than 2×1pixels.

In the present embodiment, the number of the frequency distributiongeneration units to be subjected to the binning processing is changeddepending on whether or not a component of reflected light is includedin the frequency distribution after binning processing. Therefore, forexample, in the case where the intensity of the reflected light variesdepending on the region to be measured, it is possible to reduce thenoise by the binning processing in the region where the reflected lightis weak while maintaining a high spatial resolution in the region wherethe reflected light is strong. As a result, highly accurate ranging ispossible. Further, in the present embodiment, even in the case where thenumber of the frequency distribution generation units to be subjected tothe binning processing is changed, since it is not necessary to acquirethe frequency distribution again by receiving the light again, anincrease in delay time accompanying the reacquisition of the frequencydistribution is suppressed. Therefore, according to the presentembodiment, it is possible to provide a ranging device capable ofimproving ranging accuracy while suppressing an increase in delay time.

Second Embodiment

In the first embodiment, as illustrated in FIGS. 13 to 19 , thethreshold value used for the determination of the reflected light isconstant for each bin. However, this threshold may not be constant andmay be variable from bin to bin. Hereinafter, as a second embodiment,detection of reflected light using a variable threshold value will bedescribed. Since the difference between the first embodiment and thesecond embodiment is only the operation of the determination units 343 aand 343 b, the difference will be mainly described below, and the otherpoints will be appropriately omitted or simplified.

FIG. 20 is a flowchart illustrating a peak determination processing forone pixel according to the present embodiment. The flowchart of FIG. 20differs from the flowchart of FIG. 6 in that the step S33 is replacedwith step S38.

In the step S38, the determination units 343 a and 343 b compare thethreshold value that is variable for each bin and the light receptioncount value of each bin in the frequency distribution. Thus, thedetermination units 343 a and 343 b determine whether or not thefrequency distribution output from the binning processing unit 35includes a component of reflected light, and output determinationresults. When it is determined that there is reflected light in thepixel being processed (YES in the step S38), the process proceeds to thestep S37. When it is determined that there is no reflected light in thepixel being processed (NO in the step S38), the processing proceeds tothe step S34. The variable threshold is set to a larger value for a binat an earlier time and set to a smaller value for a bin at a later time.

Next, detection of reflected light in the step S38 of FIG. 20 will bedescribed with reference to FIGS. 21 to 24 . FIG. 21 illustrates anexample of a histogram of the pixel at coordinates (0, 2) according tothe present embodiment, and FIG. 22 illustrates an example of ahistogram of the pixel at coordinates (1, 2) according to the presentembodiment. As illustrated in FIGS. 21 and 22 , the threshold value TH2has a stepwise variable value that is set to a smaller value for a binat a later time. Since the light reception count values of all the binsdo not exceed the threshold value TH2 in any of the pixels of thecoordinates, the binning processing is performed on these pixels withthe number of binning pixels of 2×1.

FIG. 23 is an example of a histogram obtained by performing binningprocessing pixels of coordinates (0, 2) and pixels of coordinates (1, 2)according to the embodiment. That is, FIG. 23 is a sum of the histogramof FIG. 21 and the histogram of FIG. 22 . In the example of FIG. 23 ,since the light reception count value of the fifth bin BN24 exceeds thethreshold value TH2, it is determined in the step S38 that reflectedlight is included. In the example of FIG. 23 , assuming that the timeintervals of the bins correspond to a distance of 20 m, the distancecorresponding to the bin BN24 exceeding the threshold value TH2 is 100m. In the example of FIG. 23 , since the light reception count value ofthe second bin BN23 does not exceed the threshold value TH2, the secondbin BN23 is not detected as reflected light.

An effect obtained by applying the variable threshold value TH2 that isset to a value larger for a bin at an earlier time and smaller for a binat a later time as in the present embodiment will be described. Ingeneral, the intensity of the reflected light from the object 40 isstronger as the object 40 is closer to the ranging device 30, and isweaker as the object 40 is farther from the ranging device 30.Therefore, by setting the threshold value of the bin at the early timecorresponding to the short distance to be large and the threshold valueof the bin at the late time corresponding to the long distance to besmall, it is possible to reduce the possibility of erroneousdetermination due to detection of light other than reflected light atthe short distance.

FIG. 24 is an example of a histogram obtained by preforming binningprocessing on the pixel of coordinates (0, 2) and the pixel ofcoordinates (1, 2) according to a comparative example of the presentembodiment. The difference from FIG. 23 is that a constant thresholdvalue TH1 is applied instead of the threshold value TH2 as in the firstembodiment. As illustrated in FIG. 24 , in this comparative example, inthe example of FIG. 24 , both the light reception count value of thesecond bin BN23 and the light reception count value of the fifth binBN24 exceed the threshold value TH1. Assuming that the light receptioncount value of the second bin BN23 is due to light other than reflectedlight, in the situation of this comparative example, erroneousdetermination may occur because the second bin BN23 exceeds thethreshold value TH1 and the light reception count value of the bin BN23is the maximum. On the other hand, in the example in which the variablethreshold value TH2 as illustrated in FIG. 23 is used, since the lightreception count value of the second bin BN23 does not exceed thethreshold value TH2, erroneous determination does not occur.

Also in this embodiment, the same effect as in the first embodiment canbe obtained. In addition, in the present embodiment, the possibility oferroneous determination can be reduced by applying a variable thresholdvalue TH2 that is set to a larger value for a bin at an earlier time anda smaller value for a bin at a later time.

It should be noted that the threshold value TH2 is not limited to a stepshape in which bins as illustrated in FIGS. 21 to 23 are divided intoseveral ranges and are set to different values for each range. Since theintensity of light is generally inversely proportional to the square ofthe distance, the threshold value TH2 in each bin may be set to adifferent value by a calculation equation having a term inverselyproportional to the square of the distance.

Third Embodiment

In the present embodiment, a specific configuration example of aphotoelectric conversion device including an avalanche photodiode whichcan be applied to the ranging device 30 according to the first or secondembodiment will be described. The configuration example of the presentembodiment is an example, and the photoelectric conversion deviceapplicable to the ranging device 30 is not limited thereto.

FIG. 25 is a schematic diagram illustrating an overall configuration ofthe photoelectric conversion device 100 according to the presentembodiment. The photoelectric conversion device 100 includes a sensorsubstrate 11 (first substrate) and a circuit substrate 21 (secondsubstrate) stacked on each other. The sensor substrate 11 and thecircuit substrate 21 are electrically connected to each other. Thesensor substrate 11 has a pixel region 12 in which a plurality of pixelcircuits 101 are arranged to form a plurality of rows and a plurality ofcolumns. The circuit substrate 21 includes a first circuit region 22 inwhich a plurality of pixel signal processing units 103 are arranged toform a plurality of rows and a plurality of columns, and a secondcircuit region 23 arranged outside the first circuit region 22. Thesecond circuit region 23 may include a circuit for controlling theplurality of pixel signal processing units 103. The sensor substrate 11has a light incident surface for receiving incident light and aconnection surface opposed to the light incident surface. The sensorsubstrate 11 is connected to the circuit substrate 21 on the connectionsurface side. That is, the photoelectric conversion device 100 is aso-called backside illumination type.

In this specification, the term “plan view” refers to a view from adirection perpendicular to a surface opposite to the light incidentsurface. The cross section indicates a surface in a directionperpendicular to a surface opposite to the light incident surface of thesensor substrate 11. Although the light incident surface may be a roughsurface when viewed microscopically, in this case, a plan view isdefined with reference to the light incident surface when viewedmacroscopically.

In the following description, the sensor substrate 11 and the circuitsubstrate 21 are diced chips, but the sensor substrate 11 and thecircuit substrate 21 are not limited to chips. For example, the sensorsubstrate 11 and the circuit substrate 21 may be wafers. When the sensorsubstrate 11 and the circuit substrate 21 are diced chips, thephotoelectric conversion device 100 may be manufactured by being dicedafter being stacked in a wafer state, or may be manufactured by beingstacked after being diced.

FIG. 26 is a schematic block diagram illustrating an arrangement exampleof the sensor substrate 11. In the pixel region 12, a plurality of pixelcircuits 101 are arranged to form a plurality of rows and a plurality ofcolumns. Each of the plurality of pixel circuits 101 includes aphotoelectric conversion unit 102 including an avalanche photodiode(hereinafter referred to as APD) as a photoelectric conversion elementin the substrate.

Of the charge pairs generated in the APD, the conductivity type of thecharge used as the signal charge is referred to as a first conductivitytype. The first conductivity type refers to a conductivity type in whicha charge having the same polarity as the signal charge is a majoritycarrier. Further, a conductivity type opposite to the first conductivitytype, that is, a conductivity type in which a majority carrier is acharge having a polarity different from that of a signal charge isreferred to as a second conductivity type. In the APD described below,the anode of the APD is set to a fixed potential, and a signal isextracted from the cathode of the APD. Accordingly, the semiconductorregion of the first conductivity type is an N-type semiconductor region,and the semiconductor region of the second conductivity type is a P-typesemiconductor region. Note that the cathode of the APD may have a fixedpotential and a signal may be extracted from the anode of the APD. Inthis case, the semiconductor region of the first conductivity type isthe P-type semiconductor region, and the semiconductor region of thesecond conductivity type is then N-type semiconductor region. Althoughthe case where one node of the APD is set to a fixed potential isdescribed below, potentials of both nodes may be varied.

FIG. 27 is a schematic block diagram illustrating a configurationexample of the circuit substrate 21. The circuit substrate 21 has thefirst circuit region 22 in which a plurality of pixel signal processingunits 103 are arranged to form a plurality of rows and a plurality ofcolumns.

The circuit substrate 21 includes a vertical scanning circuit 110, ahorizontal scanning circuit 111, a reading circuit 112, a pixel outputsignal line 113, an output circuit 114, and a control signal generationunit 115. The plurality of photoelectric conversion units 102illustrated in FIG. 26 and the plurality of pixel signal processingunits 103 illustrated in FIG. 27 are electrically connected to eachother via connection wirings provided for each pixel circuits 101.

The control signal generation unit 115 is a control circuit thatgenerates control signals for driving the vertical scanning circuit 110,the horizontal scanning circuit 111, and the reading circuit 112, andsupplies the control signals to these units. As a result, the controlsignal generation unit 115 controls the driving timings and the like ofeach unit.

The vertical scanning circuit 110 supplies control signals to each ofthe plurality of pixel signal processing units 103 based on the controlsignal supplied from the control signal generation unit 115. Thevertical scanning circuit 110 supplies control signals for each row tothe pixel signal processing unit 103 via a driving line provided foreach row of the first circuit region 22. As will be described later, aplurality of driving lines may be provided for each row. A logic circuitsuch as a shift register or an address decoder can be used for thevertical scanning circuit 110. Thus, the vertical scanning circuit 110selects a row to be output a signal from the pixel signal processingunit 103.

The signal output from the photoelectric conversion unit 102 of thepixel circuits 101 is processed by the pixel signal processing unit 103.The pixel signal processing unit 103 acquires and holds a digital signalhaving a plurality of bits by counting the number of pulses output fromthe APD included in the photoelectric conversion unit 102.

It is not always necessary to provide one pixel signal processing unit103 for each of the pixel circuits 101. For example, one pixel signalprocessing unit 103 may be shared by a plurality of pixel circuits 101.In this case, the pixel signal processing unit 103 sequentiallyprocesses the signals output from the photoelectric conversion units102, thereby providing the function of signal processing to each pixelcircuit 101.

The horizontal scanning circuit 111 supplies control signals to thereading circuit 112 based on a control signal supplied from the controlsignal generation unit 115. The pixel signal processing unit 103 isconnected to the reading circuit 112 via a pixel output signal line 113provided for each column of the first circuit region 22. The pixeloutput signal line 113 in one column is shared by a plurality of pixelsignal processing units 103 in the corresponding column. The pixeloutput signal line 113 includes a plurality of wirings, and has at leasta function of outputting a digital signal from the pixel signalprocessing unit 103 to the reading circuit 112, and a function ofsupplying a control signal for selecting a column for outputting asignal to the pixel signal processing unit 103. The reading circuit 112outputs a signal to an external storage unit or signal processing unitof the photoelectric conversion device 100 via the output circuit 114based on the control signal supplied from the control signal generationunit 115.

The arrangement of the photoelectric conversion units 102 in the pixelregion 12 may be one-dimensional. Further, the function of the pixelsignal processing unit 103 does not necessarily have to be provided oneby one in all the pixel circuits 101. For example, one pixel signalprocessing unit 103 may be shared by a plurality of pixel circuits 101.In this case, the pixel signal processing unit 103 sequentiallyprocesses the signals output from the photoelectric conversion units102, thereby providing the function of signal processing to each pixelcircuit 101.

As illustrated in FIGS. 26 and 27 , the first circuit region 22 having aplurality of pixel signal processing units 103 is arranged in a regionoverlapping the pixel region 12 in the plan view. In the plan view, thevertical scanning circuit 110, the horizontal scanning circuit 111, thereading circuit 112, the output circuit 114, and the control signalgeneration unit 115 are arranged so as to overlap a region between anedge of the sensor substrate 11 and an edge of the pixel region 12. Inother words, the sensor substrate 11 includes the pixel region 12 and anon-pixel region arranged around the pixel region 12. In the circuitsubstrate 21, the second circuit region 23 having the vertical scanningcircuit 110, the horizontal scanning circuit 111, the reading circuit112, the output circuit 114, and the control signal generation unit 115is arranged in a region overlapping with the non-pixel region in theplan view.

Note that the arrangement of the pixel output signal line 113, thearrangement of the reading circuit 112, and the arrangement of theoutput circuit 114 are not limited to those illustrated in FIG. 27 . Forexample, the pixel output signal lines 113 may extend in the rowdirection, and may be shared by a plurality of pixel signal processingunits 103 in corresponding rows. The reading circuit 112 may be providedso as to be connected to the pixel output signal line 113 of each row.

FIG. 28 is a schematic block diagram illustrating a configurationexample of one pixel of the photoelectric conversion unit 102 and thepixel signal processing unit 103 according to the present embodiment.FIG. 28 schematically illustrates a more specific configuration exampleincluding a connection relationship between the photoelectric conversionunit 102 arranged in the sensor substrate 11 and the pixel signalprocessing unit 103 arranged in the circuit substrate 21. In FIG. 28 ,driving lines between the vertical scanning circuit 110 and the pixelsignal processing unit 103 in FIG. 27 are illustrated as driving lines213 and 214.

The photoelectric conversion unit 102 includes an APD 201. The pixelsignal processing unit 103 includes a quenching element 202, a waveformshaping unit 210, a counter circuit 211, and a selection circuit 212.The pixel signal processing unit 103 may include at least one of thewaveform shaping unit 210, the counter circuit 211, and the selectioncircuit 212.

The APD 201 generates charge pairs corresponding to incident light byphotoelectric conversion. A voltage VL (first voltage) is supplied tothe anode of the APD 201. The cathode of the APD 201 is connected to afirst terminal of the quenching element 202 and an input terminal of thewaveform shaping unit 210. A voltage VH (second voltage) higher than thevoltage VL supplied to the anode is supplied to the cathode of the APD201. As a result, a reverse bias voltage that causes the APD 201 toperform the avalanche multiplication operation is supplied to the anodeand the cathode of the APD 201. In the APD 201 to which the reverse biasvoltage is supplied, when a charge is generated by the incident light,this charge causes avalanche multiplication, and an avalanche current isgenerated.

The operation modes in the case where a reverse bias voltage is suppliedto the APD 201 include a Geiger mode and a linear mode. The Geiger modeis a mode in which a potential difference between the anode and thecathode is higher than a breakdown voltage, and the linear mode is amode in which a potential difference between the anode and the cathodeis near or lower than the breakdown voltage.

The APD operated in the Geiger mode is referred to as a single photonavalanche diode (SPAD). In this case, for example, the voltage VL (firstvoltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD201 may operate in the linear mode or the Geiger mode. In the case ofthe SPAD, a potential difference becomes greater than that of the APD ofthe linear mode, and the effect of avalanche multiplication becomessignificant, so that the SPAD is preferable.

The quenching element 202 functions as a load circuit (quenchingcircuit) when a signal is multiplied by avalanche multiplication. Thequenching element 202 suppresses the voltage supplied to the APD 201 andsuppresses the avalanche multiplication (quenching operation). Further,the quenching element 202 returns the voltage supplied to the APD 201 tothe voltage VH by passing a current corresponding to the voltage dropdue to the quenching operation (recharge operation). The quenchingelement 202 may be, for example, a resistive element.

The waveform shaping unit 210 shapes the potential change of the cathodeof the APD 201 obtained at the time of photon detection, and outputs apulse signal. For example, an inverter circuit is used as the waveformshaping unit 210. Although FIG. 28 illustrates an example in which oneinverter is used as the waveform shaping unit 210, the waveform shapingunit 210 may be a circuit in which a plurality of inverters areconnected in series, or may be another circuit having a waveform shapingeffect.

The counter circuit 211 counts the pulse signals output from thewaveform shaping unit 210, and holds a digital signal indicating thecount value. When a control signal is supplied from the verticalscanning circuit 110 through the driving line 213, the counter circuit211 resets the held signal.

The selection circuit 212 is supplied with a control signal from thevertical scanning circuit 110 illustrated in FIG. 27 through the drivingline 214 illustrated in FIG. 28 . In response to this control signal,the selection circuit 212 switches between the electrical connection andthe non-connection of the counter circuit 211 and the pixel outputsignal line 113. The selection circuit 212 includes, for example, abuffer circuit or the like for outputting a signal corresponding to avalue held in the counter circuit 211.

In the example of FIG. 28 , the selection circuit 212 switches betweenthe electrical connection and the non-connection of the counter circuit211 and the pixel output signal line 113; however, the method ofcontrolling the signal output to the pixel output signal line 113 is notlimited thereto. For example, a switch such as a transistor may bearranged at a node such as between the quenching element 202 and the APD201 or between the photoelectric conversion unit 102 and the pixelsignal processing unit 103, and the signal output to the pixel outputsignal line 113 may be controlled by switching the electrical connectionand the non-connection. Alternatively, the signal output to the pixeloutput signal line 113 may be controlled by changing the value of thevoltage VH or the voltage VL supplied to the photoelectric conversionunit 102 using a switch such as a transistor.

FIG. 28 illustrates a configuration example using the counter circuit211. However, instead of the counter circuit 211, a time-to-digitalconverter (TDC) and a memory may be used to acquire a timing at which apulse is detected. In this case, the generation timing of the pulsedsignal output from the waveform shaping unit 210 is converted into adigital signal by the TDC. In this case, a control signal (referencesignal) can be supplied from the vertical scanning circuit 110illustrated in FIG. 27 to the TDC via the driving line. The TDCacquires, as a digital signal, a signal indicating a relative time ofinput timing of a pulse with respect to the control signal.

FIGS. 29A, 29B, and 29C are diagrams illustrating an operation of theAPD 201 according to the present embodiment. FIG. 29A is a diagramillustrating the APD 201, the quenching element 202, and the waveformshaping unit 210 in FIG. 28 . As illustrated in FIG. 29A, the connectionnode of the APD 201, the quenching element 202, and the input terminalof the waveform shaping unit 210 is referred to as node A. Further, asillustrated in FIG. 29A, an output side of the waveform shaping unit 210is referred to as node B.

FIG. 29B is a graph illustrating a temporal change in the potential ofnode A in FIG. 29A. FIG. 29C is a graph illustrating a temporal changein the potential of node B in FIG. 29A. During a period from time t0 totime t1, the voltage VH-VL is applied to the APD 201 in FIG. 29A. When aphoton enters the APD 201 at the time t1, avalanche multiplicationoccurs in the APD 201. As a result, an avalanche current flows throughthe quenching element 202, and the potential of the node A drops.Thereafter, the amount of potential drop further increases, and thevoltage applied to the APD 201 gradually decreases. Then, at time t2,the avalanche multiplication in the APD 201 stops. Thereby, the voltagelevel of node A does not drop below a certain constant value. Then,during a period from the time t2 to time t3, a current that compensatesfor the voltage drop flows from the node of the voltage VH to the nodeA, and the node A is settled to the original potential at the time t3.

In the above-described process, the potential of node B becomes the highlevel in a period in which the potential of node A is lower than acertain threshold value. In this way, the waveform of the drop of thepotential of the node A caused by the incidence of the photon is shapedby the waveform shaping unit 210 and output as a pulse to the node B.

The light receiving units 341 a and 341 b in the first or secondembodiment correspond to, for example, the APD 201, the quenchingelement 202, and the waveform shaping unit 210 of the presentembodiment.

According to the present embodiment, a photoelectric conversion deviceusing an avalanche photodiode which can be applied to the ranging device30 of the first or second embodiment is provided.

Fourth Embodiment

FIG. 30 is a block diagram of a photodetection system according to thepresent embodiment. More specifically, FIG. 30 is a block diagram of adistance image sensor and a light source device as an example of theranging device 30 described in the above embodiments.

As illustrated in FIG. 30 , the distance image sensor 401 includes anoptical system 402, a photoelectric conversion device 403, an imageprocessing circuit 404, a monitor 405, and a memory 406. The distanceimage sensor 401 receives light (modulated light or pulsed light)emitted from a light source device 411 toward an object and reflected bythe surface of the object. The distance image sensor 401 can acquire adistance image corresponding to a distance to the object based on a timeperiod from light emission to light reception. The light source devicecorresponds to the light emitting unit 32 of the above embodiments, andthe photoelectric conversion device 403 corresponds to other blocks inthe ranging device 30.

The optical system 402 includes one or a plurality of lenses, and guidesimage light (incident light) from the object to the photoelectricconversion device 403 to form an image on a light receiving surface(sensor portion) of the photoelectric conversion device 403.

The photoelectric conversion device 403 supplies a distance signalindicating a distance obtained from the received light signal to theimage processing circuit 404. The image processing circuit 404 performsimage processing for forming a distance image based on the distancesignal supplied from the photoelectric conversion device 403. Thedistance image (image data) obtained by the image processing can bedisplayed on the monitor 405 and stored (recorded) in the memory 406.

The distance image sensor 401 configured in this manner can acquire anaccurate distance image by applying the configuration of theabove-described embodiments.

Fifth Embodiment

FIGS. 31A and 31B are block diagrams of equipment relating to anin-vehicle ranging device according to the present embodiment. Equipment80 includes a distance measurement unit 803, which is an example of theranging device of the above-described embodiments, and a signalprocessing device (processing device) that processes a signal from thedistance measurement unit 803. The equipment 80 includes the distancemeasurement unit 803 that measures a distance to an object, and acollision determination unit 804 that determines whether or not there isa possibility of collision based on the measured distance. The distancemeasurement unit 803 is an example of a distance information acquisitionunit that obtains distance information to the object. That is, thedistance information is information on a distance to the object or thelike. The collision determination unit 804 may determine the collisionpossibility using the distance information.

The equipment 80 is connected to a vehicle information acquisitiondevice 810, and can obtain vehicle information such as a vehicle speed,a yaw rate, and a steering angle. Further, the equipment 80 is connectedto a control ECU 820 which is a control device that outputs a controlsignal for generating a braking force to the vehicle based on thedetermination result of the collision determination unit 804. Theequipment 80 is also connected to an alert device 830 that issues analert to the driver based on the determination result of the collisiondetermination unit 804. For example, when the collision possibility ishigh as the determination result of the collision determination unit804, the control ECU 820 performs vehicle control to avoid collision orreduce damage by braking, returning an accelerator, suppressing engineoutput, or the like. The alert device 830 alerts the user by sounding analarm, displaying alert information on a screen of a car navigationsystem or the like, or giving vibration to a seat belt or a steeringwheel. These devices of the equipment 80 function as a movable bodycontrol unit that controls the operation of controlling the vehicle asdescribed above.

In the present embodiment, ranging is performed in an area around thevehicle, for example, a front area or a rear area, by the equipment 80.FIG. 31B illustrates equipment when ranging is performed in the frontarea of the vehicle (ranging area 850). The vehicle informationacquisition device 810 as a ranging control unit sends an instruction tothe equipment 80 or the distance measurement unit 803 to perform theranging operation. With such a configuration, the accuracy of distancemeasurement can be further improved.

Although the example of control for avoiding a collision to anothervehicle has been described above, the embodiment is applicable toautomatic driving control for following another vehicle, automaticdriving control for not going out of a traffic lane, or the like.Furthermore, the equipment is not limited to a vehicle such as anautomobile and can be applied to a movable body (movable apparatus) suchas a ship, an airplane, a satellite, an industrial robot and a consumeruse robot, or the like, for example. In addition, the equipment can bewidely applied to equipment which utilizes object recognition orbiometric authentication, such as an intelligent transportation system(ITS), a surveillance system, or the like without being limited tomovable bodies.

Modified Embodiments

The present invention is not limited to the above embodiment, andvarious modifications are possible. For example, an example in whichsome of the configurations of any one of the embodiments are added toother embodiments and an example in which some of the configurations ofany one of the embodiments are replaced with some of the configurationsof other embodiments are also embodiments of the present invention.

The disclosure of this specification includes a complementary set of theconcepts described in this specification. That is, for example, if adescription of “A is B” (A=B) is provided in this specification, thisspecification is intended to disclose or suggest that “A is not B” evenif a description of “A is not B” (A B) is omitted. This is because it isassumed that “A is not B” is considered when “A is B” is described.

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2022-112260, filed Jul. 13, 2022, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A ranging device comprising: a time counting unitconfigured to perform time counting; a plurality of light receivingunits each configured to generate a signal including a pulse based onincident light and perform an operation of counting the pulse; aplurality of frequency distribution generation units arrangedcorresponding to the plurality of light receiving units and eachconfigured to hold a frequency distribution including a light receptioncount value of the pulse acquired at predetermined time intervals in thetime counting; a binning processing unit configured to perform binningprocessing for one or more frequency distributions held in one or morefrequency distribution generation units of the plurality of frequencydistribution generation units; a determination unit configured todetermine whether a component of reflected light from an object isincluded in the frequency distribution subjected to the binningprocessing; a binning setting unit configured to change the number ofthe frequency distribution generation units to be subjected to thebinning processing performed by the binning processing unit inaccordance with a result of the determination by the determination unit;and a distance calculation unit configured to calculate a distance tothe object based on a time interval corresponding to a reception time ofthe reflected light when the determination unit determines that acomponent of the reflected light is included in the frequencydistribution subjected to the binning processing.
 2. The ranging deviceaccording to claim 1 further comprising: a light emitting unitconfigured to emit light to the object; and a control unit configured tosynchronously control a timing at which the light emitting unit emitslight and a timing at which the time counting unit starts time counting.3. The ranging device according to claim 1, wherein each of theplurality of light receiving units includes a plurality of photoelectricconversion elements and integrates to count pulses based on lightincident on each of the plurality of photoelectric conversion elements.4. The ranging device according to claim 1, wherein when thedetermination unit determines that a component of the reflected light isnot included in the frequency distribution subjected to the binningprocessing, the binning setting unit increases the number of thefrequency distribution generation units to be subjected to the binningprocessing by the binning processing unit.
 5. The ranging deviceaccording to claim 4, wherein after the binning setting unit increasesthe number of the frequency distribution generation units to besubjected to the binning processing by the binning processing unit, thebinning processing unit performs the binning processing again.
 6. Theranging device according to claim 5, wherein the determination unitdetermines again whether a component of reflected light from the objectis included in the frequency distribution subjected to the binningprocessing by the binning processing unit again.
 7. The ranging deviceaccording to claim 1, wherein the determination unit determines that acomponent of the reflected light is included when a light receptioncount value at any one of a plurality of time intervals is larger than athreshold value.
 8. The ranging device according to claim 7, wherein thethreshold value is set for each of the plurality of time intervals andset to a smaller value as the elapsed time from the start of the timecounting is longer.
 9. The ranging device according to claim 8, whereinthe threshold value is set to include a term inversely proportional tothe square of the elapsed time for each of a plurality of timeintervals.
 10. The ranging device according to claim 1, wherein thebinning setting unit sets a different number of frequency distributiongeneration units to be subjected to the binning processing by thebinning processing unit between a part of the plurality of frequencydistribution generation units and another part of the plurality offrequency distribution generation units.
 11. The ranging deviceaccording to claim 1, wherein the distance calculation unit outputsinformation indicating that the distance cannot be calculated when thedetermination unit determines that a component of the reflected light isnot included in the frequency distribution subjected to the binningprocessing and the number of the frequency distribution generation unitssubjected to the binning processing by the binning processing unitreaches a predetermined maximum value.
 12. A photodetection systemcomprising: the ranging device according to claim 1; and a signalprocessing unit configured to process a signal output from the rangingdevice.
 13. A movable body comprising: the ranging device according toclaim 1; and a movable body control unit configured to control themovable body based on distance information acquired by the rangingdevice.